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The associated random walk and martingales 
in random walks with stationary increments 

D. R. Grey^ 



Abstract 

We extend the notion of the associated random walk and the Wald 
martingale in random walks where the increments are independent and 
identically distributed to the more general case of stationary ergodic 
increments. Examples are given where the increments are Markovian or 
Gaussian, and an application in queueing is considered. 

AMS subject classification (MSC2010) 60G50, 60G42, 60G10, 60K25 



1 Introduction and definition 

Let Xi, X2, ... be independent identically distributed (i.i.d.) random 
variables with distribution function (d.f.) F and positive mean. If S n = 
X\ + X2 + ■ ■ ■ + X n for each n = 0, 1, 2, . . . then the process {S n } 
is called the random walk with increments X±, X%, .... Because the 
increments have positive mean, by the strong law of large numbers the 
random walk will in the long run drift upwards to infinity 
There may exist 8^0 such that 

/oo 
e~ ex dF{x) = 1, 
-00 

in which case 9 is unique, and necessarily positive because of the upward 
drift. In this case, if we define a new increment distribution by 

dF*{x) := e~ ex dF(x), 
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then we obtain the associated random walk, which has downward drift. 
Because of the definition, probabilities in one random walk may easily 
be expressed in terms of those in the other. For instance, renewal theory 
in the associated random walk yields the Cramer estimate of the prob- 
ability of ruin in the original random walk, the parameter 9 determining 
the rate of exponential decay (Feller (1971), XI. 7, XII. 6; see also As- 
musscn (2000)). Note also that since F*{-6) = e 0x dF*{x) = 1 the 
association is a duality relationship in the sense that if we perform the 
analogous transformation on the associated random walk with 9 replaced 
by —9 then we obtain the original one. 

For the same value of 9, it may easily be shown that V n := e~ BSn 
defines a martingale, known as the Wald martingale. This is also useful 
in the investigation of hitting probabilities. 

The question arises to what extent the concepts of the associated ran- 
dom walk and the Wald martingale may be generalized to the case where 
the increments are no longer necessarily i.i.d. but merely stationary and 
ergodic. In such cases, because of the ergodic theorem, the random walk 
still drifts upwards to infinity and so we might still be interested in, for 
example, the probability of ruin (hitting a low level). The following is 
a suggested way forward. It should be noted that, once one goes bey- 
ond the ergodic theorem, the general stationary ergodic process behaves 
quite differently from the independent-increments case. For example, the 
convergence rate in the ergodic theorem may be arbitrarily slow, however 
many moments may be finite. For background, see for example Eberlcin 
& Taqqu (1986). 

The work below is motivated by that of Lu (1991) on branching pro- 
cesses in random environments (Smith & Wilkinson (1969), Athreya 
& Karlin (1971)) and by a convergence result given a straightforward 
proof by the author (Grey (2001)). We obtain generalizations of the as- 
sociated random walk and the Wald martingale, under certain assump- 
tions. Three applications are considered in Section 2, to the Markov and 
Gaussian cases, and to a queueing problem. Our work is also relevant to 
random walks in random environments; see for example Revesz (1990), 
Part III. 

To proceed, we need to make the following assumptions. In Section 
2 we shall show that these assumptions are satisfied in some important 
cases of interest. 



Associated random walk and martingales 
Assumption 1 There exists 9 > such that 

q : = lim E( e - 8S ") 

n— t-oo 

exists and is positive and finite. 



This assumption is trivially satisfied in the i.i.d. case, with as iden- 
tified earlier and q = 1. It is important to note that since S n — > oo 
with probability one, there can be at most one value of 9 satisfying the 
assumption. This is because, if such 9 exists, for any positive constant 
K 

E{e- 0S --S n <-K)^q 

as the remaining contribution to the expectation tends to zero, by the 
bounded convergence theorem. From here it is easy to see that if < 

<j)<9, 

limsup£(e-* 5 ";S„ < -K) < e'^'^q 

n— t-oo 

whence by similar reasoning 

limsup£(e-* 5 ") < e-V-^Kq 

n— t-oo 

and so, since K is arbitrary, E(e~ <t ' Sn ) — > 0. Similarly if (f> > then 
E(e-^) -> oo. 

For our next assumption we extend our sequence of increments to a 
doubly infinite one . . . , X_ i , Xq , X\ , . . . , as is always possible with a 
stationary sequence (Breiman (1968), Proposition 6.5). We also define 
the more general partial sum 

n 

Sm ,n • — ^ ^ X r . 

r—ra 

Let T m ^n denote the cr-field generated by {X r ; r = m, . . . , n}. 
Assumption 2 For all k = 1, 2, . . . and for all B 6 J--k,k, 
q(B):= lim E(e~ es - m - n ; B) 

m,n— t-oo 

exists, where 9 is as defined in Assumption 1. 

Again we refer immediately to the i.i.d. case, where the assumption is 
easily seen to be satisfied, the limiting operation being essentially trivial, 
and we may write explicitly 

q(B) = E(e- 9S - k - k ;B). 
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If we now fix k and define for m, n> k 

E(e-™-™.»;B) 
r m ,nK D ) ■ £?( e -«IS- ra ,„) 

for B G J--k.k, then clearly P^ n is a probability measure on J--k,k- 
Moreover if Assumptions 1 and 2 hold then 

P^ n {B) — > as m,n — > oo 

for all _B G F-k.k- We make use of the result, given a straightforward 
proof in Grey (2001), rather simpler than for the more general case of 
signed measures considered by Halmos (1950, p. 170), and a special case 
of the Vitali-Hahn-Saks theorem (Dunford & Schwartz (1958), III.7.2- 
4), that if {P n } is a sequence of probability measures on a space (fi, J 7 ) 
such that the limit P{A) := lim^oo P n (A) exists for all A G J 7 , then P 
is a probability measure on (ft, J 7 ). It follows that 

P*(B) :=5*p 

defines a probability measure on J--k,k- Since this definition obviously 
does not depend upon k, we have consistency between different values 
of k and therefore a probability measure defined on IJfcLi ~^-k,k- The 
Caratheodory extension theorem (Durrett (1996), Appendix A. 2) now 
ensures that P* can be extended to a probability measure defined on 
the whole cr-field T := ^-00,00- 

It is the probability measure P* which we use to define the distribu- 
tion of the increments of the associated random walk. Note that, because 
the original process is stationary and because of the double-ended lim- 
iting process involved in the definition of P* , the associated process of 
increments is also stationary; it would not have been possible to achieve 
this with a single-ended sequence. Whether the associated process is 
necessarily ergodic and whether duality occurs is left open here; some 
further remarks are given in Section 3. Note that ergodicity and duality 
do occur in the two special cases studied in detail in Section 2. Note 
also that in the i.i.d. case the associated random walk as defined here 
coincides with the one which we have already met, since, for example, 
in the discrete case 

P*{X 1 =x u ...,X k = x k ) = e-° £?=i Xi P{X 1 =x u ...,X k =x k ) 

k 

= Hie-^PiX, = Xi )) 
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as expected. 

To construct our martingale, we need to replace Assumption 2 by the 
following one-sided equivalent, which again is to be read in conjunction 
with Assumption 1. Write 74 '■= 74, fc- 

Assumption 2* For all k = 1, 2, . . . and for all B e 74, 

r(B) := lim E{e- 0S ™;B) 

n— yoc 

exists, where 9 is as defined in Assumption 1. 

If this assumption holds, then by the aforementioned convergence the- 
orem, for each k, r is a measure on 74 with total mass q. Also it is 
absolutely continuous with respect to P, since 

P{B) = => / e- 0S ~ dP = => r{B) = 0. 
Jb 

So r restricted to 74 has a Radon-Nikodym derivative 14 with respect 
to P: 

I V k dP = r{B) for all B G Jfc, 
is 

where 14 is J4-measurable. 

But if B £ 74, then £> e 74+1, and therefore 

/ Vfe = r(B) = / V fc+ i dP for all B E T k . 
Jb J b 

So, by definition of conditional expectation, V k = E(Vk+i \ 74) almost 
surely, which shows that {14} is a martingale with respect to {74}- 

A^oie In many cases it will be true that 14 = lim„^oo E(e~ eSn |74) al- 
most surely, but it seems difficult to try to use this equation as a defin- 
ition of 14 in general. 

In the case of i.i.d. increments it is easy to see that 14 = e~ eSk almost 
surely, and so our definition generalizes that of the Wald martingale. 



2 Three examples 

In this section we demonstrate the existence of the associated random 
walk in two important cases of interest: stationary Markov chain incre- 
ments and stationary Gaussian increments. In both cases, as indeed in 
the simpler i.i.d. case, certain regularity conditions will be required. The 
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corresponding martingale is mentioned more briefly in each case. We 
also consider an application in queueing theory. 

2.1 Stationary Markov chain increments 

Here we suppose that the increments {X n } perform a stationary irre- 
ducible (and therefore ergodic) aperiodic Markov chain with countable 
state space S. We shall use labels such as i and j to represent the actual 
sizes of the increments, so that they need not be integer- valued or non- 
negative; however this will not prevent us from also using them to denote 
positions in vectors and matrices, since this non-standard notation will 
not lead to confusion. The associated Markov chain constructed here has 
been considered in a rather more general context by, for example, Arjas 
& Speed (1973). 

Let the transition matrix of the Markov chain be P — (pij) and let 
its equilibrium distribution be given by the column vector ir — (iti). 
Note that if 1 denotes the vector consisting entirely of ones, then P has 
Perron-Frobenius eigenvalue 1 with ir T and 1 as corresponding left and 
right eigenvectors respectively; also 7r T l = 1 and P n — > ln T as n — > oo. 

The regularity conditions which we impose are as follows. For some 9 > 

the matrix Q with elements (pije~ e i) has Perron-Frobenius eigenvalue 

1 and corresponding left and right eigenvectors v T (with components 
(vi)) and c (with components (cj)) respectively; also v T c = 1 and Q n — > 
cv T as n — > oo. This requirement is not especially stringent when the 
state space S is finite; the Perron-Frobenius eigenvalue X(6) of Q for 
general 8 behaves rather like the Laplace transform F(0) in the i.i.d. case 
(Lu (1991)). 

We firstly show that Assumption 1 holds, with 9 as defined above. 

= iv T Q n l — > tt t cv t 1 as n —> oo. 

To check Assumption 2, we shall evaluate 

E ( e - es - m - n ;X_ k = i_k,...,X k = i k ) 

for given k < m, n and i-k, ■ ■ ■ , ik £ S. By the Markov property, 
this may be written as the product of the three factors e - e (*-*H I-**) 
P(A_ fe = i_fc,...,A fe = ik) together with E(e~ eSk+1 - n \X k = i k ) and 
E(e~ 0S - m -- k - 1 \X- k = i-k)- The first factor may be written 

-9(i- k +-+i k ) . 
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The second factor may be written 

7 j ■ ■ ■ 7 j e flklk + l ■ ■ -Pln-Hn- 

ik+ieS !„GS 

Because the reverse Markov chain has transition probabilities irjPji/iTi, 
the third factor may be written 

V •••V c -eri-*-i+-+i- m ) 7r »-*-i D . 

The second factor is seen to be the ik component of the vector Q n h l 
and so converges to c ik v T l as n —¥ oo. 

Writing ^u, := nie^ 6 " 1 for each i e S and letting be the corresponding 
vector, after cancellation and rearrangement the third factor is seen to be 
/i^ 1 times the component of the vector fi T Q m ~ k and so converges 
to fi~} Vi_ k (i T c as m — > oo. Note that 



= niPije-^cj = -■Yi'-j' " !< -j = ma = -* T c- 

jeSies ies jes zes 

Using this fact and putting all the preceding results together, after can- 
cellation we see that P*(X-k = i_k, . . . ,Xk = ik) exists and is equal 
to 

-0(i_ fc+ iH hik) n . . n r ii- 

e Pi- k i- k+1 ■ ■ -Pik-^k^k u i-k ■ 

Writing p*- :— p^e~ Q ^ Cj / 'ci and w* :— CiVi for each i, j 6 S, the above 
may also be written 

P*(X- k =i_ k ,...,X k = i k ) = n*_ k p*_ ki _ k+1 ...Pi k _ lik . 

It is a routine matter to check that the numbers {p*j) form the transition 
probabilities of a Markov chain and that (tt*) is an equilibrium distri- 
bution for it. We have thus established that the associated random walk 
exists and its increments perform a stationary Markov chain, which is 
also obviously irreducible and aperiodic like the original. Duality is left 
as an exercise. 

The martingale may be constructed similarly. Letting B = {X\ = 
ii,...,Xk = ik} it may be calculated that 

E( e - es ";B) -> P(B)e- e(n+ -+^c lk v T l as n -> oo 

and so Assumption 2* holds; moreover, since B is an atom of the a-field 
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J-fc it follows that Vk = e~ eSk Cx k v T l. We may take v T l = 1 since v 
and c have so far only been scaled relative to each other. This gives the 
martingale 14 = Cx k e~ eSk which has also been used by Lu (1991). 

2.2 Stationary Gaussian increments 

Now let {X n } be a stationary Gaussian process in which each X n has 
normal distribution with mean p > and variance a 2 > 0. For each 
r = 1, 2, . . . let p r be the correlation coefficient between X n and X n+r 
for any n. These parameters completely determine the behaviour of the 
process, since the joint distribution of any finite collection of the X n is 
multivariate normal. 

The regularity condition we need here is that 

oo 
r=l 

This is an asymptotic independence property more than sufficient for er- 
godicity, and one which is easily satisfied by commonly studied processes 
such as autoregressive and moving average processes. 

Under this condition, let R := Y^T=i P r anc ^ & : ~ ^2rLi r P r '^ these 
will both be finite. Below we shall see that R > — \ necessarily, and that 
we need to exclude the extreme case R = —\. 

We firstly find 9 such that Assumption 1 is satisfied. Since S n = 
X\+- ■ -+X n has a normal distribution with mean n[i and variance a 2 [n+ 
2 X^™=i ( n ~~ r )Pr\i by the standard formula for the Laplace transform of 
the normal distribution we have that 

E{e~ eSn ) = exp|-n M 6i+ icr 2 [n + 2^(n -r)p r ]0 2 j . 
Under our regularity condition 

^^(n — r)p r = nR — S + o(l) as n->oo 

r=l 

and so in particular 

var S n = a 2 {n[l + 2R] - 2S) + o(l) as n -)■ oo, 

whence 1 + 2R > 0. It is possible to construct examples with 1 + 2R = 
(such as X n := p, + Z n — Z n -i where {Z n } are i.i.d. ./V(0, \a 2 ) random 



Associated random walk and martingales 



9 



variables) but if we exclude this rather extreme case then we see that 
convergence of E(e~ eSn ) to a positive limit will occur if and only if 

-fi6+ \a 2 \\ + 2R}6 2 = 0. 

This yields 

a 2 [l + 2i?] 

and then it is easy to compute that for this value of 6 

-es x f 1 

E(e ^ Me ^|____| as 

We turn to Assumption 2. Fix k, and take 6* as just identified. For 
m, n > k it is evidently relevant to look at the distribution of Y := 
S-m-k-i + Sk+i, n conditional on A_ fc = x_ fe , . . . , X k = x k , or X = x 
say. If the unconditional distribution of Y is denoted by N(v,t 2 ), the 
vector of covariances of Y with the components of X is denoted by 
v, and the mean and covariance matrix of X are denoted by /x and S 
respectively, then by multivariate normal theory (Mardia, Kent & Bibby 
(1979), Theorem 3.2.4), the conditional distribution is N (v + v T 'E~ 1 (x- 
/j,), t 2 — v T 'E~ 1 v). (For ease of notation, we suppress the dependence of 
Y, v, t 2 and v on m and n.) 

A typical component of v is of the form 




for some i, and so v converges to a finite limit as to, n — > oo. Also 
v = (m + n — 2k)fi. Then, since 

var S- m , n = var (1 T X + Y) = 1 T S1 + 21 T v + t 2 , 

we can use the estimate of var S n obtained in checking Assumption 1 to 
deduce that 

r 2 -cr 2 [l + 2 J R](TO + n+l) 

converges to a finite limit as m, n — >• oo. Putting these results together 
we see that 

E(e- eY \X = x) = exp {-[u + v T S- 1 (x - p)]8 + \[t 2 - v T T.- 1 v]e 2 } 

converges to a positive limit as m, n — > oo, since because of the value of 
8 the difference between the large terms in v and r 2 converges to a finite 
limit, and the other terms also converge to finite limits. We may denote 
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the limit in the above by exp(ot T x + (3) for some constants a and f3. It 
is then easy to see that for any B £ JF-k.k, 



E( e ~ es - m ' n ;B) E(exp{-91 T X + a T X + j3}; B) as m, n oo, 



and we have established Assumption 2. It is not hard in this case to see 
that the associated random walk has the same covariance structure as 
the original, but downward drift — /i. 

Calculations similar to the above may be used to check that Assump- 
tion 2* holds in this case also, and that the associated martingale is of the 
form Vk = cxp( / y T X + S) for some 7, 5, where now X = (X\, . . . , Xk) T . 



Suppose we have a G/GI/1 queue in which the inter-arrival times {T n } 
form a stationary ergodic sequence and the independent service times 
{[/„} are i.i.d. with ET n > EU n > 0. Then the waiting times {W n } 
satisfy 



and it follows by a standard argument, dating back to Lindley (1952) in 
the case of i.i.d. inter-arrival times and exploited, among others, by King- 
man (1964), that W n has an equilibrium distribution which is the same 
as the distribution of minus the all-time minimum of an unrestricted ran- 
dom walk started at zero in state zero, with increments X n := T- n —U- n . 
The tail of this distribution is therefore intimately related to the prob- 
ability of ruin in this random walk, and, in particular, the parameter 9, 
if it exists, has an important part to play. 

The simplest example is the M/M/l queue where the T n are inde- 
pendent exponential with parameter A and the U n are exponential with 
parameter /1, where /i > A > 0. In this case the X n are i.i.d. and 
E(e~ eXn ) = A^t/{(A + 6)(fi — 6)}, which is easily seen to be equal to 
one when 9 = fi — A. So the key parameter 9 depends upon both the 
arrival rate and the service rate in a simple and obvious way. 

As another example, suppose that the U n have some arbitrary dis- 
tribution with Laplace transform (f)(9) — E(e~ eUrl ), and that there is 
a regular appointments system such that customer n arrives at clock 
time A _1 n + e„, where {e„} is a sequence of i.i.d. errors with Laplace 
transform i}j(9) — E(e~ 6erl ). In this case it is possible to compute 



2.3 A queueing application 



W n+1 = (W n + U, 



n 




E(e- es ") = [(f>(-9)] n cxp(-\- 1 n9)^(9)^(-9) 
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and so the key parameter 9 satisfies the equation 

<£(-<?) exp^A" 1 !?) = 1 

which does not involve the distribution of the e n . By considering the 
special case </>(#) = (i/(ti + 0), so that the U n are exponentially distrib- 
uted, and the mean inter-arrival time and mean service time are A -1 
and [i~ x respectively as in the previous M/M/l example, it is possible 
to compare an appointments system with random (Poisson) arrivals. For 
the value 9 = /i — A found in the case of random arrivals, we may com- 
pute <j>(— 9) exp(— A~ 1 9) = (/i/A)exp(l — (/ti/A)) < 1, and so the actual 
value of the key parameter 9 for the appointments system must be larger 
than for random arrivals. This suggests a thinner tail for the equilibrium 
waiting time distribution, and a more efficient system. 



3 Some remarks on duality and asymptotic 
independence 

If we wish for duality to occur, then, replacing 9 by —6 and denoting 
expectation with respect to P* by E* , we require for B G T-k,k that 

E *r e 0S- m , n . B \ 

Now, denoting E(e r - s ) by q r ^ s and noting that P* s has Radon- 
Nikodym derivative q~le~ ss ~ r ' s with respect to P, we have that 



E*( e 0S -^-B) = f 

J E 

! 

r,s-KxJ B 

= lim / 

r,s^r<xJ B 



Li i ) i / e es - m '"dP: 



e 8S — q-le- 0S — dP 
= q~ 1 lim / e - 9S -r,-™-i e -es n+ i,s dp 

r.s^co J B 

For the required convergence to occur, it seems therefore that for large 
m and n there must be approximate independence between SL ri _ m _i, 
S n +i,s and the event B, so that we can say that the above integral 
is approximately P(B)E(e~ es - r '- m - 1 )E(e~ eSrl + 1 ' s ) which converges to 
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P(B)q 2 as r, s — > oo. Hence under these circumstances 
E*(e es -^-B) q- 1 P{B)q 2 



P(£>) as m, n — > oo. 



The asymptotic independence is a kind of mixing condition which is 
already stronger than ergodicity, suggesting that the latter is not the 
most appropriate property to be considering in this context. See Bradley 
(2005) on mixing conditions. 
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